Any discussion of the background art throughout the specification should in no way be considered as an admission that such art is widely known or forms part of common general knowledge in the field.
A significant issue with embedding eye-tracking technology into mobile devices is the significant amount of power required to illuminate the eye sufficiently in order to create a bright reflection from the cornea surface able to be detected by an eye tracking algorithm under conditions of normal use for a hand-held device such as a mobile phone including at different distances from the eye, use in the outdoors, or whilst wearing sunglasses. This power requirement may be in the order of several watts which makes eye-tracking a prohibitively inefficient feature.
The need to power illumination devices to allow for proper eye-tracking system operation has significant draw backs. For example, light emitting diodes (LEDS) are only about 20% efficient at converting electrons into infrared photons, the rest of the energy is dissipated as heat. Beyond draining the battery quickly, the LEDs have the secondary effect of heating the mobile device, which is undesirable for many reasons. Specifically, when used in a hot vehicle cabin, the additional heating of the LEDs can push the device to failure or reduced efficiency unless expensive heat sinks are added.
In addition for eye tracking, it is desirable to maximize the ratio of the controlled IR light from illumination sources relative to the uncontrolled environmental light such as the sun or other sources of light such as lamps or displays. The higher this ratio, the less likely it is for an eye tracking algorithm to fail to locate the reflection on the cornea correctly. The sun in particular creates very strong reflections on the eye and so when tracking the eye outdoors, it is desirable to have very bright light illuminating the eye in order to combat these environmental reflections.
A first form of sensor, called a “rolling shutter” sensor (also known as a progressive scan sensor), operates by exposing each row of pixels in the image sequentially. This requires the illuminators to be powered (active) whilst each row of pixels on the sensor is integrating light. Consequently the illuminators are required to be active for the entire frame period in order to expose the whole image correctly. For example if the sensor is producing a new image every 33 ms (30 Hz) the illuminators will need to be active for the entire 33 ms, even if each pixel only requires 5 ms of time in order to integrate photons sufficiently.
Another form of sensor, called a “global-shutter” sensor, the entire array of pixels is imaged simultaneously, which is desirable from a power saving perspective as it allows the illuminated light to be pulsed in a short-duration high-intensity flash. Referring to the example above, the illuminators would only need to be active for the 5 ms pixel integration time. Therefore global shutter sensors offer significant power saving advantages for eye tracking.
However there are many other aspects (including complexity, size, cost, sensitivity and noise performance) as to why global-shutter sensors are considered inferior to progressive-scan sensors for mobile device designs.
US Patent Application Publication 2010/0066975 to Rehnstrom entitled “Eye Tracking Illumination” makes some preliminary suggestions on how to contain power usage in eye tracking systems and also suggests controlling the beam direction when illuminating eyes. However, Rehnstrom does not consider the operation of different camera sensors and does not specifically take into account the distance of a subject's eyes from the camera/illuminator. Accordingly, only limited power reduction can be achieved.